Data-Dependent Analysis of Learning Algorithms

This thesis studies the generalization ability of machine learning algorithms in a statistical setting. It focuses on the data-dependent analysis of the generalization performance of learning algorithms in order to make full use of the potential of the actual training sample from which these algorithms learn.¶ First, we propose an extension of the standard framework for the derivation of generalization bounds for algorithms taking their hypotheses from random classes of functions. ... ¶ Second, we study in more detail generalization bounds for a specific algorithm which is of central importance in learning theory, namely the Empirical Risk Minimization algorithm (ERM). ..

On the importance of small coordinate projections

It has been recently shown that sharp generalization bounds can be obtained when the function class from which the algorithm chooses its hypotheses is “small” in the sense that the Rademacher averages of this function class are small. We show that a new more general principle guarantees good generalization bounds. The new principle requires that random coordinate projections of the function class evaluated on random samples are “small” with high probability and that the random class of functions allows symmetrization. As an example, we prove that this geometric property of the function class is exactly the reason why the two lately proposed frameworks, the luckiness (Shawe-Taylor et al., 1998) and the algorithmic luckiness (Herbrich and Williamson, 2002), can be used to establish generalization bounds

Accurate splice site prediction using support vector machines

Proceeding

POIMs: positional oligomer importance matrices—understanding support vector machine-based signal detectors

Motivation: At the heart of many important bioinformatics problems, such as gene finding and function prediction, is the classification of biological sequences. Frequently the most accurate classifiers are obtained by training support vector machines (SVMs) with complex sequence kernels. However, a cumbersome shortcoming of SVMs is that their learned decision rules are very hard to understand for humans and cannot easily be related to biological facts

Soluble CD36 Ectodomain Binds Negatively Charged Diacylglycerol Ligands and Acts as a Co-Receptor for TLR2

BACKGROUND:Cluster of differentiation 36 (CD36) is a transmembrane glycoprotein involved in many biological processes, such as platelet biology, angiogenesis and in the aetiopathology of atherosclerosis and cardiovascular diseases. Toll-like receptors (TLRs) are one of the most important receptors of the innate immune system. Their main function is the recognition of conserved structure of microorganisms. This recognition triggers signaling pathways that activate transcription of cytokines and co-stimulatory molecules which participate in the generation of an immune response against microbes. In particular, TLR2 has been shown to recognize a broad range of ligands. Recently, we showed that CD36 serves as a co-receptor for TLR2 and enhances recognition of specific diacylglycerides derived from bacteria. METHODOLOGY/ PRINCIPAL FINDINGS:Here, we investigate the mechanism by which CD36 contributes to ligand recognition and activation of TLR2 signaling pathway. We show that the ectodomain of murine CD36 (mCD36ED) directly interacts with negatively charged diacylglycerol ligands, which explains the specificity and selectivity of CD36 as a TLR2 co-receptor. We also show that mCD36ED amplifies the pro-inflammatory response to lipoteichoic acid in macrophages of wild-type mice and restores the pro-inflammatory response of macrophages from mice deficient in CD36 (oblivious), but not from mice deficient in cluster of differentiation 14 (CD14) (heedless). CONCLUSION/ SIGNIFICANCE: These data indicate that the CD36 ectodomain is the only relevant domain for activation of TLR2 signaling pathway and that CD36 and CD14 have a non-redundant role for loading ligands onto TLR2 in the plasma-membrane. The pro-inflammatory role of soluble CD36 can be relevant in the activation of the immune response against pathogens, as well as in the progression of chronic diseases. Therefore, an increased level of soluble forms of CD36, which has been reported to be increased in type II diabetic patients, could accelerate atherosclerosis by increasing the pro-inflammatory response to diacylglycerol ligands

25th annual computational neuroscience meeting: CNS-2016

The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong

Data-Dependent Analysis of Learning Algorithms

Typeset in Computer Modern by TEX and LATEX 2ε. Except where otherwise indicated, this thesis is my own original work. The results in this thesis were produced under the supervision of Shahar Mendel-son and Bob Williamson, and partly in collaboration with Peter Bartlett. The main contribution of this thesis are two related parts. The main technical results in the first part on random subclass bounds appeared as a journal paper with Shahar Mendelson [1], and an earlier conference paper [2]. The results were discussed with my supervisors Shahar Mendelson and Bob Williamson, who gave me advice and direction. The re-sults on the data-dependent estimation of localized complexities for the Empirical Risk Minimization algorithm appeared as part of a conference paper with Peter Bartlett and Shahar Mendelson [3], and the optimality results are work in progress and contained in an unpublished manuscript with Peter Bartlett and Shahar Mendelson [4]. This second part of the thesis is based on intensive discussions and technical advice from Shaha

Random subclass bounds

Abstract. It has been recently shown that sharp generalization bounds can be obtained when the function class from which the algorithm chooses its hypotheses is “small ” in the sense that the Rademacher averages of this function class are small [8, 9]. Seemingly based on different arguments, generalization bounds were obtained in the compression scheme [7], luckiness [13], and algorithmic luckiness [6] frameworks in which the “size ” of the function class is not specified a priori. We show that the bounds obtained in all these frameworks follow from the same general principle, namely that coordinate projections of this function subclass evaluated on random samples are “small ” with high probability